Add Tickbox and Math Protection #39

zkamvar · 2021-05-01T00:24:58Z

Description

This fixes TWO issues with one solution!

What is the solution?

added the @asis attribute in the xml definition that allows text to be passed from xml to markdown without being escaped
added find_between(): a function that searches between patterns (used to find nodes between both $$ and $).
added md_ns(): a named alias for the commonmark namespace
added protect_math(): a function for protecting math elements (not default)
added tickbox protection for GitHub checkboxes/tasklists
added add_node_siblings(), an internal function that will take a nodelist and add them as siblings following a given node in the document with the option to have the siblings replace the node in question.
added procedure for splitting text nodes, recombining them, and adding them to the document.

Related Issue

This addresses #38 by allowing dollar-notation LaTeX math to be protected (though bracket notation is still needed). It also fixes #10 and supersedes #20.

Example

library("tinkr")
m <- yarn$new(system.file("extdata", "math-example.md", package = "tinkr"))
m$head(15) # math formatting broken (tickboxes in tact!)
#> ---
#> title: An example with math elements
#> ---
#> 
#> This is cheap, it only costs 10$!
#> 
#> This example has $\\LaTeX$ elements embedded in the
#> text. It is intended to demonstrate that m $\\alpha\_\\tau$ h
#> mode can work with tinkr. $y =
#> mx + b$
#> 
#> - [ ] This is an empty checkbox
#> - [x] This is a checked checkbox
#> - [This is a link](https://ropensci.org)
#> - \[this is an example\]

m$protect_math()$head(15) # formatting rescued!
#> ---
#> title: An example with math elements
#> ---
#> 
#> This is cheap, it only costs 10$!
#> 
#> This example has $\LaTeX$ elements embedded in the
#> text. It is intended to demonstrate that m $\alpha_\tau$ h
#> mode can work with tinkr. $y =
#> mx + b$
#> 
#> - [ ] This is an empty checkbox
#> - [x] This is a checked checkbox
#> - [This is a link](https://ropensci.org)
#> - \[this is an example\]

m$tail() # block math protected!
#> 
#> $$
#> Q_{N(norm)}=\frac{C_N +C_{N-1}}2\times
#> \frac{\sum _{i=N-n}^{N}Q_i} {\sum_{j=N-n}^{N}{(\frac{C_j+C_{j-1}}2)}}
#> $$

^{Created on 2021-05-07 by the reprex package (v2.0.0)}

^{Created on 2021-05-06 by the reprex package (v2.0.0)}

This is definitely still a work in progress!

commonmark wraps text in <emph></emph> tags. If I add `asis` to them, they disappear... which probably means that I just need to add the `asis` tag to the text nodes :face_palm:

It's now gibberish!

This is why when you run into odd things during development, to restart and work with a clean environment, because sometimes you screw up.

This allows us to include the delimiters for the search

Deer lord, I hope this makes sense six months down the line

I had modified `find_between()` from pegboard:::find_between_tags() (https://github.com/carpentries/pegboard/blob/378c627b4e08869540aa049b678f9c4a560dff59/R/div.R#L125-L159) My initial modification to extract all of the descendants was in the context of block math where I knew that I just needed to add the 'asis' attribute to the nodes. In the context of inline math, where I needed to create new nodes, this was not feasible since extracting descendant-or-self would give me duplicate nodes, which would hinder the process of creating new nodes and lead to stuttering of the output. Separating out the search functionality into `find_between()` really helped me fix those two inconsistencies

maelle

This is brilliant! 👏 Thank you!

Apart from my small comments/questions, would it make sense to add a paragraph about Math in the README?
Also, when would one not want to use protect_math()?

NEWS.md

R/add_md.R

R/asis-nodes.R

NEWS.md

Co-authored-by: Maëlle Salmon <maelle.salmon@yahoo.se>

Add not math dollar in example

zkamvar · 2021-05-07T17:10:32Z

Apart from my small comments/questions, would it make sense to add a paragraph about Math in the README?

Yes, it would!

Also, when would one not want to use protect_math()?

I can really only think of edge cases (e.g. writing a BASIC tutorial or describing the Burroughs-Wheeler Transform with ^ and $ as the delimiters.)

zkamvar · 2021-05-07T18:50:53Z

README.md

+### LaTeX equations
+
+While markdown parsers like pandoc know what LaTeX is, commonmark does
+not, and that means LaTeX equations will end up with extra markup due to
+commonmark’s desire to escape characters.
+
+For users of the `yarn` object, if you have LaTeX equations that use
+either `$` or `$$` to delimit them, you can protect them from formatting
+changes with the `$protect_math()` method:
+
+``` r
+path <- system.file("extdata", "math-example.md", package = "tinkr")
+math <- tinkr::yarn$new(path)
+math$tail() # malformed
+#| 
+#| $$
+#| Q\_{N(norm)}=\\frac{C\_N +C\_{N-1}}2\\times
+#| \\frac{\\sum *{i=N-n}^{N}Q\_i} {\\sum*{j=N-n}^{N}{(\\frac{C\_j+C\_{j-1}}2)}}
+#| $$
+math$protect_math()$tail() # success!
+#| 
+#| $$
+#| Q_{N(norm)}=\frac{C_N +C_{N-1}}2\times
+#| \frac{\sum _{i=N-n}^{N}Q_i} {\sum_{j=N-n}^{N}{(\frac{C_j+C_{j-1}}2)}}
+#| $$
+```
+
+Note, however, that there are a few caveats for this:
+
+1.  The dollar notation for inline math must be adjacent to the text.
+    E.G. `$\alpha$` is valid, but `$ \alpha$` and `$\alpha $` are not
+    valid.
+
+2.  We do not currently have support for bracket notation
+
+3.  If you use a postfix dollar sign in your prose (e.g. BASIC commands
+    or a Burroughs-Wheeler Transformation demonstration), you must be
+    sure to either use punctuation after the trailing dollar sign OR
+    format the text as code. (i.e. `` `INKEY$` `` is good, but `INKEY$`
+    by itself is not good and will be interepreted as LaTeX code,
+    throwing an error:
+
+    ``` r
+    path <- system.file("extdata", "basic-math.md", package = "tinkr")
+    math <- tinkr::yarn$new(path)
+    math$head(15) # malformed
+    #| ---
+    #| title: basic math
+    #| ---
+    #| 
+    #| BASIC programming can make things weird:
+    #| 
+    #| - Give you $2 to tell me what INKEY$ means.
+    #| - Give you $2 to *show* me what INKEY$ means.
+    #| - Give you $2 to *show* me what `INKEY$` means.
+    #| 
+    #| Postfix dollars mixed with prefixed dollars can make things weird:
+    #| 
+    #| - We write $2 but say 2$ verbally.
+    #| - We write $2 but *say* 2$ verbally.
+    math$protect_math() #error
+    #| Error: Inline math delimiters are not balanced.
+    #| 
+    #| HINT: If you are writing BASIC code, make sure you wrap variable
+    #|       names and code in backtics like so: `INKEY$`. 
+    #| 
+    #| Below are the pairs that were found:
+    #|            start...end
+    #|            -----...---
+    #|  Give you $2 to ... me what INKEY$ means.
+    #|  Give you $2 to ... 2$ verbally.
+    #| We write $2 but ...
+    ```
+


@maelle, let me know if this looks okay for you for the README

it does! I just wonder what folks who do not use the yarn object should do: probably update their code to use the yarn object?

I think the reason why I am using the yarn object here is because we can use our namespace object in the internal parsing, but I can export that object so that we can use it as default.

structure(c(md = "http://commonmark.org/xml/1.0"), class = "xml_namespace")

maelle

Thanks for all the updates!! I only added a small comment.

zkamvar · 2021-05-11T14:31:52Z

Excellent! I realized I need to add a couple of more touches to make it complete (e.g. clearing the namespace of the new nodes before copying to avoid the issue of duplicating namespace).

Thank you so much for reviewing this!

- add xml_ns_strip() before node insertion events to avoid duplicate namespaces from accumulating (see https://community.rstudio.com/t/adding-nodes-in-xml2-how-to-avoid-duplicate-default-namespaces/84870) - export md_ns() and protect_math() for users of to_xml() - protect_tickbox() is now default in to_xml() - updated README - updated asis tests

add asis node; demonstrate math protection

28cd450

This is definitely still a work in progress!

zkamvar mentioned this pull request May 1, 2021

Characters transformed into mathematical expressions #38

Closed

zkamvar added 16 commits May 1, 2021 18:38

fix example

22f4464

add emph asis type

11d946b

commonmark wraps text in <emph></emph> tags. If I add `asis` to them, they disappear... which probably means that I just need to add the `asis` tag to the text nodes :face_palm:

update asis node example

acffb34

add asis node testing

6ad6945

simplify code and parse out reusable chunks

6c9af60

store arguments in yarn object for resetting.

e7846c0

make more progress towards inline math.

365b861

It's now gibberish!

use correct variable names

44e05fb

This is why when you run into odd things during development, to restart and work with a clean environment, because sometimes you screw up.

add grouping for full regex

55e16a7

update protect_math() and snapshot

e6d501a

update documentation

14655c5

add include = TRUE for block math

282c360

This allows us to include the delimiters for the search

update regex for inline math

fe955ee

update inline math parsing

831c27f

Deer lord, I hope this makes sense six months down the line

add new test with snapshot

60d9a53

document; add condidtions for no math

72a69b1

zkamvar marked this pull request as ready for review May 6, 2021 00:43

zkamvar marked this pull request as draft May 6, 2021 00:47

zkamvar requested a review from maelle May 6, 2021 15:55

zkamvar added 3 commits May 6, 2021 12:03

update test

b058508

add nocov tags

14e0cee

zkamvar marked this pull request as ready for review May 6, 2021 19:12

zkamvar added 3 commits May 6, 2021 17:25

add tickbox protection

21d1688

document

c9f711d

remove unneeded body

df7cf0d

zkamvar changed the title ~~add asis node; demonstrate math protection~~ Add Tickbox and Math Protection May 7, 2021

add not math dollar in example

284bff0

maelle approved these changes May 7, 2021

View reviewed changes

zkamvar commented May 7, 2021

View reviewed changes

NEWS.md Outdated Show resolved Hide resolved

zkamvar and others added 5 commits May 7, 2021 08:02

Apply suggestions from code review

aae7811

Co-authored-by: Maëlle Salmon <maelle.salmon@yahoo.se>

Merge pull request #40 from ropensci/small

589642c

Add not math dollar in example

update error message

0701022

update documentation; delta internal function name

47e4d84

add test for error

d46e9ef

zkamvar added 4 commits May 7, 2021 11:36

make tickboxes protected by default

c2748ed

update README to explain math

b9fd529

escape protection early if nothing needs protectin

5060438

process tickboxes up front

c00a12a

zkamvar commented May 7, 2021

View reviewed changes

zkamvar requested a review from maelle May 7, 2021 21:55

maelle approved these changes May 10, 2021

View reviewed changes

zkamvar merged commit bd76a24 into master May 11, 2021

zkamvar deleted the wear-your-mathk branch May 11, 2021 16:13

zkamvar mentioned this pull request May 11, 2021

Add replace() method for yarn object #41

Open

zkamvar mentioned this pull request Feb 4, 2022

Local Building lesson fail carpentries/sandpaper#249

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add Tickbox and Math Protection #39

Add Tickbox and Math Protection #39

zkamvar commented May 1, 2021 •

edited

Loading

maelle left a comment

zkamvar commented May 7, 2021

zkamvar May 7, 2021

maelle May 10, 2021

zkamvar May 11, 2021 •

edited

Loading

maelle left a comment

zkamvar commented May 11, 2021

Add Tickbox and Math Protection #39

Add Tickbox and Math Protection #39

Conversation

zkamvar commented May 1, 2021 • edited Loading

Description

Related Issue

Example

maelle left a comment

Choose a reason for hiding this comment

zkamvar commented May 7, 2021

zkamvar May 7, 2021

Choose a reason for hiding this comment

maelle May 10, 2021

Choose a reason for hiding this comment

zkamvar May 11, 2021 • edited Loading

Choose a reason for hiding this comment

maelle left a comment

Choose a reason for hiding this comment

zkamvar commented May 11, 2021

zkamvar commented May 1, 2021 •

edited

Loading

zkamvar May 11, 2021 •

edited

Loading